<html>
<head><meta charset="utf-8"><title>Expanding macros in `world_symbols` · t-compiler/rust-analyzer · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/index.html">t-compiler/rust-analyzer</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html">Expanding macros in `world_symbols`</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="193465046"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193465046" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Timo Freiberg <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193465046">(Apr 09 2020 at 15:20)</a>:</h4>
<p>I'm slowly working on <a href="https://github.com/rust-analyzer/rust-analyzer/issues/3851" title="https://github.com/rust-analyzer/rust-analyzer/issues/3851">https://github.com/rust-analyzer/rust-analyzer/issues/3851</a> on the side and I don't want to spam the Github issue anymore, so I'm writing my questions down here.</p>
<p>Issues I was stuck on for a while but then solved (maybe someone will find those interesting?):</p>
<ol>
<li>
<p>A freshly created <code>Semantics</code> in the <a href="https://github.com/rust-analyzer/rust-analyzer/blob/baf9fcc38ebd5a52b22fdf115a7475f9002058f6/crates/ra_ide_db/src/symbol_index.rs#L369" title="https://github.com/rust-analyzer/rust-analyzer/blob/baf9fcc38ebd5a52b22fdf115a7475f9002058f6/crates/ra_ide_db/src/symbol_index.rs#L369">leaf call</a> failed to expand the macro because its cache was empty.<br>
    -&gt; then I noticed that the parent function <a href="https://github.com/rust-analyzer/rust-analyzer/blob/baf9fcc38ebd5a52b22fdf115a7475f9002058f6/crates/ra_ide_db/src/symbol_index.rs#L102" title="https://github.com/rust-analyzer/rust-analyzer/blob/baf9fcc38ebd5a52b22fdf115a7475f9002058f6/crates/ra_ide_db/src/symbol_index.rs#L102"><code>file_symbols</code></a> actually parses the file via <code>db.parse</code>. Creating a <code>Semantics</code> at the start of the function and calling <code>sema.parse</code> instead filled the cache. Passing that <code>sema</code> instance down worked.</p>
</li>
<li>
<p><code>sema.expand</code> returns a <code>SyntaxNode</code>, but to recursively call <a href="https://github.com/rust-analyzer/rust-analyzer/blob/baf9fcc38ebd5a52b22fdf115a7475f9002058f6/crates/ra_ide_db/src/symbol_index.rs#L320" title="https://github.com/rust-analyzer/rust-analyzer/blob/baf9fcc38ebd5a52b22fdf115a7475f9002058f6/crates/ra_ide_db/src/symbol_index.rs#L320"><code>source_file_to_file_symbols</code></a> I need a <code>SourceFile</code><br>
    -&gt; or actually, I just need something to call <code>.syntax()</code> on, so an <code>AstNode</code><br>
    -&gt; or actually, I can just change the signature of <code>source_file_to_file_symbols</code> to take the <code>SyntaxNode</code> that <code>.syntax()</code> returns... <span aria-label="face palm" class="emoji emoji-1f926" role="img" title="face palm">:face_palm:</span></p>
</li>
</ol>



<a name="193465296"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193465296" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193465296">(Apr 09 2020 at 15:23)</a>:</h4>
<p>I wonder if we shouldn't start building the index from HIR instead</p>



<a name="193465382"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193465382" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Timo Freiberg <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193465382">(Apr 09 2020 at 15:23)</a>:</h4>
<p>Would that change the approach completely?</p>



<a name="193465415"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193465415" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193465415">(Apr 09 2020 at 15:23)</a>:</h4>
<p>yes, pretty much</p>



<a name="193465793"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193465793" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Timo Freiberg <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193465793">(Apr 09 2020 at 15:26)</a>:</h4>
<p>Hm, I'm interested yet I'm not even sure how to express my questions <span aria-label="sweat smile" class="emoji emoji-1f605" role="img" title="sweat smile">:sweat_smile:</span><br>
I guess the current approach is AST-based?</p>



<a name="193465835"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193465835" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193465835">(Apr 09 2020 at 15:26)</a>:</h4>
<p>I think it's something that we do want to do at some point, and it might not be worth the effort to try to adapt the current approach to expand macros</p>



<a name="193465847"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193465847" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193465847">(Apr 09 2020 at 15:26)</a>:</h4>
<p>yes</p>



<a name="193465914"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193465914" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193465914">(Apr 09 2020 at 15:27)</a>:</h4>
<p>not sure what <span class="user-mention" data-user-id="133169">@matklad</span> thinks though, I haven't really dealt with the symbol index so far</p>



<a name="193465949"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193465949" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193465949">(Apr 09 2020 at 15:27)</a>:</h4>
<p>Yeah, I think we are roughtly at the position where hir-based index is worthwhile.</p>
<p>I think the only pre-requesite we need to do is this:</p>



<a name="193466006"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193466006" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193466006">(Apr 09 2020 at 15:27)</a>:</h4>
<p>Instead of completely forgetting the cfged-out items, we should still collect them, but in a "disabled" state</p>



<a name="193466121"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193466121" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193466121">(Apr 09 2020 at 15:28)</a>:</h4>
<p>so then to build the index you'd walk through all the modules pretty similarly to how we do it in <a href="https://github.com/rust-analyzer/rust-analyzer/blob/master/crates/rust-analyzer/src/cli/analysis_stats.rs" title="https://github.com/rust-analyzer/rust-analyzer/blob/master/crates/rust-analyzer/src/cli/analysis_stats.rs">analysis-stats</a>, I guess</p>



<a name="193466154"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193466154" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193466154">(Apr 09 2020 at 15:28)</a>:</h4>
<p>Uhm..... Now that I think about it, it might be the case that what we want is an index based on the item-tree IR</p>



<a name="193466431"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193466431" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Timo Freiberg <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193466431">(Apr 09 2020 at 15:30)</a>:</h4>
<p>For what it's worth, I think a not-very-pretty version of including macro-expanded code in the index is not far off, but I'm very new to the codebase so there are lots of unknown unknowns</p>



<a name="193466578"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193466578" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193466578">(Apr 09 2020 at 15:31)</a>:</h4>
<p>Agree that bolting on macro-expansion <em>for the current crate</em> on top of our index is not that much of work</p>



<a name="193466622"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193466622" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193466622">(Apr 09 2020 at 15:31)</a>:</h4>
<p>IIRC, I did that at one point, in the very first implementation of macro expansion</p>



<a name="193485333"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193485333" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Timo Freiberg <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193485333">(Apr 09 2020 at 17:48)</a>:</h4>
<p>so, do you think it's worth it to work on the current index? or should that effort rather be focused on redesigning it?</p>



<a name="193723727"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Expanding%20macros%20in%20%60world_symbols%60/near/193723727" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Timo Freiberg <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Expanding.20macros.20in.20.60world_symbols.60.html#193723727">(Apr 12 2020 at 18:16)</a>:</h4>
<p>I continued trying to bolt on macro-expansion for a bit until I got stuck remapping the text range. My motivation to continue is not very high, partly because I feel like it might be better to work on a hir-based index. Is there a ticket for that?</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>